[inference provider] Add wavespeed.ai as an inference provider #1424

arabot777 · 2025-05-05T02:28:49Z

What’s in this PR
WaveSpeedAI is a high-performance AI image and video generation service platform, offering industry-leading generation speeds. Now, want to be listed as an Inference Provider on the Hugging Face Hub

The JS Client Integration was completed based on the inference-providers help documentation and passed the test. I am submitting the pr now and look forward to further communication with you

Test

pnpm --filter @huggingface/inference test "test/InferenceClient.spec.ts" -t "^Wavespeed AI"

> @huggingface/[email protected] test /Users/shanliu/work/huggingface.js/packages/inference
> vitest run --config vitest.config.mts "test/InferenceClient.spec.ts"


 RUN  v0.34.6 /Users/shanliu/work/huggingface.js/packages/inference

 ✓ test/InferenceClient.spec.ts (104) 198160ms
   ✓ InferenceClient (104) 198160ms
     ✓ backward compatibility (1)
       ✓ works with old HfInference name
     ↓ HF Inference (49) [skipped]
       ↓ throws error if model does not exist [skipped]
       ↓ fillMask [skipped]
       ↓ works without model [skipped]
       ↓ summarization [skipped]
       ↓ questionAnswering [skipped]
       ↓ tableQuestionAnswering [skipped]
       ↓ documentQuestionAnswering [skipped]
       ↓ documentQuestionAnswering with non-array output [skipped]
       ↓ visualQuestionAnswering [skipped]
       ↓ textClassification [skipped]
       ↓ textGeneration - gpt2 [skipped]
       ↓ textGeneration - openai-community/gpt2 [skipped]
       ↓ textGenerationStream - meta-llama/Llama-3.2-3B [skipped]
       ↓ textGenerationStream - catch error [skipped]
       ↓ textGenerationStream - Abort [skipped]
       ↓ tokenClassification [skipped]
       ↓ translation [skipped]
       ↓ zeroShotClassification [skipped]
       ↓ sentenceSimilarity [skipped]
       ↓ FeatureExtraction [skipped]
       ↓ FeatureExtraction - auto-compatibility sentence similarity [skipped]
       ↓ FeatureExtraction - facebook/bart-base [skipped]
       ↓ FeatureExtraction - facebook/bart-base, list input [skipped]
       ↓ automaticSpeechRecognition [skipped]
       ↓ audioClassification [skipped]
       ↓ audioToAudio [skipped]
       ↓ textToSpeech [skipped]
       ↓ imageClassification [skipped]
       ↓ zeroShotImageClassification [skipped]
       ↓ objectDetection [skipped]
       ↓ imageSegmentation [skipped]
       ↓ imageToImage [skipped]
       ↓ imageToImage blob data [skipped]
       ↓ textToImage [skipped]
       ↓ textToImage with parameters [skipped]
       ↓ imageToText [skipped]
       ↓ request - openai-community/gpt2 [skipped]
       ↓ tabularRegression [skipped]
       ↓ tabularClassification [skipped]
       ↓ endpoint - makes request to specified endpoint [skipped]
       ↓ endpoint - makes request to specified endpoint - alternative syntax [skipped]
       ↓ chatCompletion modelId - OpenAI Specs [skipped]
       ↓ chatCompletionStream modelId - OpenAI Specs [skipped]
       ↓ chatCompletionStream modelId Fail - OpenAI Specs [skipped]
       ↓ chatCompletion - OpenAI Specs [skipped]
       ↓ chatCompletionStream - OpenAI Specs [skipped]
       ↓ custom mistral - OpenAI Specs [skipped]
       ↓ custom openai - OpenAI Specs [skipped]
       ↓ OpenAI client side routing - model should have provider as prefix [skipped]
     ↓ Fal AI (4) [skipped]
       ↓ textToImage - black-forest-labs/FLUX.1-schnell [skipped]
       ↓ textToImage - SD LoRAs [skipped]
       ↓ textToImage - Flux LoRAs [skipped]
       ↓ automaticSpeechRecognition - openai/whisper-large-v3 [skipped]
     ↓ Featherless (3) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
       ↓ textGeneration [skipped]
     ↓ Replicate (10) [skipped]
       ↓ textToImage canonical - black-forest-labs/FLUX.1-schnell [skipped]
       ↓ textToImage canonical - black-forest-labs/FLUX.1-dev [skipped]
       ↓ textToImage canonical - stabilityai/stable-diffusion-3.5-large-turbo [skipped]
       ↓ textToImage versioned - ByteDance/SDXL-Lightning [skipped]
       ↓ textToImage versioned - ByteDance/Hyper-SD [skipped]
       ↓ textToImage versioned - playgroundai/playground-v2.5-1024px-aesthetic [skipped]
       ↓ textToImage versioned - stabilityai/stable-diffusion-xl-base-1.0 [skipped]
       ↓ textToSpeech versioned [skipped]
       ↓ textToSpeech OuteTTS -  usually Cold [skipped]
       ↓ textToSpeech Kokoro [skipped]
     ↓ SambaNova (3) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
       ↓ featureExtraction [skipped]
     ↓ Together (4) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
       ↓ textToImage [skipped]
       ↓ textGeneration [skipped]
     ↓ Nebius (3) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
       ↓ textToImage [skipped]
     ↓ 3rd party providers (1) [skipped]
       ↓ chatCompletion - fails with unsupported model [skipped]
     ↓ Fireworks (2) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
     ↓ Hyperbolic (4) [skipped]
       ↓ chatCompletion - hyperbolic [skipped]
       ↓ chatCompletion stream [skipped]
       ↓ textToImage [skipped]
       ↓ textGeneration [skipped]
     ↓ Novita (2) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
     ↓ Black Forest Labs (2) [skipped]
       ↓ textToImage [skipped]
       ↓ textToImage URL [skipped]
     ↓ Cohere (2) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
     ↓ Cerebras (2) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
     ↓ Nscale (3) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
       ↓ textToImage [skipped]
     ↓ Groq (2) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
     ↓ OVHcloud (4) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
       ↓ textGeneration [skipped]
       ↓ textGeneration stream [skipped]
     ✓ Wavespeed AI (5) 89033ms
       ✓ textToImage - wavespeed-ai/flux-schnell 89032ms
       ✓ textToImage - wavespeed-ai/flux-dev-lora 12369ms
       ✓ textToImage - wavespeed-ai/flux-dev-lora-ultra-fast 17936ms
       ✓ textToVideo - wavespeed-ai/wan-2.1/t2v-480p 79507ms
       ✓ imageToImage - wavespeed-ai/hidream-e1-full 74481ms

 Test Files  1 passed (1)
      Tests  5 passed | 103 skipped (108)
   Start at  14:33:17
   Duration  89.62s (transform 315ms, setup 14ms, collect 368ms, tests 89.03s, environment 0ms, prepare 74ms)

SBrandeis

Hello, thank you for your contribution
The code is of great quality overall - I left a few comments regarding our code style.
Please make sure the client can be used to query your API for all supported tasks, and that the payload are matching your API.
Thanks again!

SBrandeis · 2025-05-19T14:36:21Z

packages/inference/README.md

+- [HF Inference API (serverless)](https://huggingface.co/models?inference=warm&sort=trending)



Suggested change

- [HF Inference API (serverless)](https://huggingface.co/models?inference=warm&sort=trending)

has deleted

SBrandeis · 2025-05-19T14:41:26Z

packages/inference/test/InferenceClient.spec.ts

+					hfModelId: "wavespeed-ai/wan-2.1/i2v-480p",
+					providerId: "wavespeed-ai/wan-2.1/i2v-480p",
+					status: "live",
+					task: "image-to-video",


this task is not supported in the client code - let's remove it for now

has deleted

SBrandeis · 2025-05-19T14:43:28Z

packages/inference/src/providers/wavespeed-ai.ts

+import { InferenceOutputError } from "../lib/InferenceOutputError";
+import { ImageToImageArgs } from "../tasks";
+import type { BodyParams, HeaderParams, RequestArgs, UrlParams } from "../types";
+import { delay } from "../utils/delay";
+import { omit } from "../utils/omit";
+import { base64FromBytes } from "../utils/base64FromBytes";
+import {
+	TaskProviderHelper,
+	TextToImageTaskHelper,
+	TextToVideoTaskHelper,
+	ImageToImageTaskHelper,
+} from "./providerHelper";
+


We use import type when the import is only used as a type

Suggested change

import { InferenceOutputError } from "../lib/InferenceOutputError";

import { ImageToImageArgs } from "../tasks";

import type { BodyParams, HeaderParams, RequestArgs, UrlParams } from "../types";

import { delay } from "../utils/delay";

import { omit } from "../utils/omit";

import { base64FromBytes } from "../utils/base64FromBytes";

import {

TaskProviderHelper,

TextToImageTaskHelper,

TextToVideoTaskHelper,

ImageToImageTaskHelper,

} from "./providerHelper";

import { InferenceOutputError } from "../lib/InferenceOutputError";

import type { ImageToImageArgs } from "../tasks";

import type { BodyParams, HeaderParams, RequestArgs, UrlParams } from "../types";

import { delay } from "../utils/delay";

import { omit } from "../utils/omit";

import { base64FromBytes } from "../utils/base64FromBytes";

import type {

TaskProviderHelper,

TextToImageTaskHelper,

TextToVideoTaskHelper,

ImageToImageTaskHelper,

} from "./providerHelper";

Modify as suggested

SBrandeis · 2025-05-19T15:08:19Z

packages/inference/src/providers/wavespeed-ai.ts

+	};
+}
+
+type WaveSpeedAIResponse<T = WaveSpeedAITaskResponse> = WaveSpeedAICommonResponse<T>;


I'm not sure this type alias is needed, can we remove it?

Suggested change

type WaveSpeedAIResponse<T = WaveSpeedAITaskResponse> = WaveSpeedAICommonResponse<T>;

WaveSpeedAICommonResponse can be renamed to WaveSpeedAIResponse

This type is needed and will be used in two places. It's uncertain whether it will be used again in the future.
It follows the DRY (Don't Repeat Yourself) principle
It provides better type safety (through default generic parameters)
It makes the code more readable and maintainable

SBrandeis · 2025-05-19T15:20:13Z

packages/inference/src/providers/wavespeed-ai.ts

+				case "completed": {
+					// Get the video data from the first output URL
+					if (!taskResult.outputs?.[0]) {
+						throw new InferenceOutputError("No video URL in completed response");
+					}
+					const videoResponse = await fetch(taskResult.outputs[0]);
+					if (!videoResponse.ok) {
+						throw new InferenceOutputError("Failed to fetch video data");
+					}
+					return await videoResponse.blob();


From what I understand, the payload can be something else than a video (eg an image)
Let's update the error message to reflect that

yes,
I revised it.

SBrandeis · 2025-05-19T15:24:51Z

packages/inference/src/providers/wavespeed-ai.ts

+		if (!args.parameters) {
+			return {
+				...args,
+				model: args.model,
+				data: args.inputs,
+			};
+		} else {
+			return {
+				...args,
+				inputs: base64FromBytes(
+					new Uint8Array(args.inputs instanceof ArrayBuffer ? args.inputs : await (args.inputs as Blob).arrayBuffer())
+				),
+			};
+		}
+	}
+
+	override preparePayload(params: BodyParams): Record<string, unknown> {
+		return {
+			...omit(params.args, ["inputs", "parameters"]),
+			...(params.args.parameters as Record<string, unknown>),
+			image: params.args.inputs,
+		};
+	}


I think only one of the two ( preparePayload or preparePayloadAsync) should be responsible for building the payload, meaning, I'd rather move the rename of inputs to image in preparePayloadAsync an have preparePayload as dumb as possible

cc @hanouticelina - would love your opinion on that specific point

I only kept preparePayloadAsync func

I think only one of the two ( preparePayload or preparePayloadAsync) should be responsible for building the payload, meaning, I'd rather move the rename of inputs to image in preparePayloadAsync an have preparePayload as dumb as possible

yes agree!

SBrandeis · 2025-05-19T15:27:55Z

packages/inference/src/providers/wavespeed-ai.ts

+				inputs: base64FromBytes(
+					new Uint8Array(args.inputs instanceof ArrayBuffer ? args.inputs : await (args.inputs as Blob).arrayBuffer())
+				),


Does the wavespeed API support base64-encoded images as inputs?

hanouticelina

thank you @arabot777 for the PR! I left some minor comments. I tested the 3 tasks supported by Wavespeed.ai and it works as expected with the changes I suggested.

hanouticelina · 2025-05-20T14:38:07Z

packages/inference/src/lib/getProviderHelper.ts

@@ -47,6 +47,7 @@ import type {
 import * as Replicate from "../providers/replicate";
 import * as Sambanova from "../providers/sambanova";
 import * as Together from "../providers/together";
+import * as WavesppedAI from "../providers/wavespeed-ai";


Suggested change

import * as WavesppedAI from "../providers/wavespeed-ai";

import * as WavespeedAI from "../providers/wavespeed-ai";

hanouticelina · 2025-05-20T14:38:23Z

packages/inference/src/lib/getProviderHelper.ts

+		"text-to-image": new WavesppedAI.WavespeedAITextToImageTask(),
+		"text-to-video": new WavesppedAI.WavespeedAITextToVideoTask(),
+		"image-to-image": new WavesppedAI.WavespeedAIImageToImageTask(),


Suggested change

"text-to-image": new WavesppedAI.WavespeedAITextToImageTask(),

"text-to-video": new WavesppedAI.WavespeedAITextToVideoTask(),

"image-to-image": new WavesppedAI.WavespeedAIImageToImageTask(),

"text-to-image": new WavespeedAI.WavespeedAITextToImageTask(),

"text-to-video": new WavespeedAI.WavespeedAITextToVideoTask(),

"image-to-image": new WavespeedAI.WavespeedAIImageToImageTask(),

hanouticelina · 2025-05-21T09:36:21Z

packages/inference/src/providers/wavespeed-ai.ts

+	constructor() {
+		super(WAVESPEEDAI_API_BASE_URL);
+	}
+


you should override preparePayload to avoid sending unexpected fields to the API, since we rely on preparePayloadAsync to format the payload for this task

Suggested change

override preparePayload(params: BodyParams): Record<string, unknown> {

return params.args;

}

hanouticelina · 2025-05-21T09:36:26Z

packages/inference/src/providers/wavespeed-ai.ts

+	async preparePayloadAsync(args: ImageToImageArgs): Promise<RequestArgs> {
+		return {
+			...args,
+			inputs: args.parameters?.prompt,


based on https://wavespeed.ai/models/wavespeed-ai/hidream-e1-full, it seems that the prompt should be send in "prompt" not "inputs". let me know if i'm missing something here.

Suggested change

inputs: args.parameters?.prompt,

prompt: args.parameters?.prompt,

arabot777 added 2 commits May 5, 2025 09:45

add wavespeed.ai as an inference provider

a4d8504

delete debug log

686931e

arabot777 requested review from julien-c, hanouticelina and SBrandeis as code owners May 5, 2025 02:28

arabot777 and others added 6 commits May 5, 2025 17:31

Merge branch 'main' into feat/wavespeedai

0e71b88

Merge branch 'main' into feat/wavespeedai

4461225

Merge branch 'main' into feat/wavespeedai

e0bf580

Merge branch 'main' into feat/wavespeedai

07af35f

Merge branch 'main' into feat/wavespeedai

fa3afa4

support lora

214ff99

SBrandeis reviewed May 19, 2025

View reviewed changes

hanouticelina added the inference-providers integration of a new or existing Inference Provider label May 20, 2025

arabot777 and others added 4 commits May 20, 2025 21:31

code review

47c64c6

Merge branch 'main' into feat/wavespeedai

7270c5c

code review

ba35791

Merge branch 'main' into feat/wavespeedai

ca35eab

arabot777 requested a review from SBrandeis May 20, 2025 13:44

arabot777 and others added 2 commits May 20, 2025 21:48

delete unused import

80d4640

Merge branch 'main' into feat/wavespeedai

77be0c6

hanouticelina reviewed May 21, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[inference provider] Add wavespeed.ai as an inference provider #1424

[inference provider] Add wavespeed.ai as an inference provider #1424

arabot777 commented May 5, 2025 •

edited

Loading

SBrandeis left a comment

SBrandeis May 19, 2025

arabot777 May 20, 2025

SBrandeis May 19, 2025

arabot777 May 20, 2025

SBrandeis May 19, 2025

arabot777 May 20, 2025

SBrandeis May 19, 2025

arabot777 May 20, 2025

SBrandeis May 19, 2025

arabot777 May 20, 2025

SBrandeis May 19, 2025

arabot777 May 20, 2025

hanouticelina May 20, 2025

SBrandeis May 19, 2025

arabot777 May 20, 2025

hanouticelina left a comment

hanouticelina May 20, 2025

hanouticelina May 20, 2025

hanouticelina May 21, 2025

hanouticelina May 21, 2025

		- [HF Inference API (serverless)](https://huggingface.co/models?inference=warm&sort=trending)

	import * as WavesppedAI from "../providers/wavespeed-ai";
	import * as WavespeedAI from "../providers/wavespeed-ai";

	inputs: args.parameters?.prompt,
	prompt: args.parameters?.prompt,

[inference provider] Add wavespeed.ai as an inference provider #1424

Are you sure you want to change the base?

[inference provider] Add wavespeed.ai as an inference provider #1424

Conversation

arabot777 commented May 5, 2025 • edited Loading

SBrandeis left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hanouticelina left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arabot777 commented May 5, 2025 •

edited

Loading